Supplementary Material for Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks

A core technical contribution of this work is the generalization of the Gaussian permutohedral lattice convolution proposed in [1] to the full non-separable case with the ability to perform backpropagation. Although, conceptually, there are minor difference between non-Gaussian and general parameterized filters, there are non-trivial practical differences in terms of the algorithmic implementation. The Gauss filters belong to the separable class and can thus be decomposed into multiple sequential one dimensional convolutions. We are interested in the general filter convolutions, which can not be decomposed. Thus, performing a general permutohedral convolution at a lattice point requires the computation of the inner product with the neighboring elements in all the directions in the high-dimensional space. Here, we give more details of the implementation differences of separable and non-separable filters. In the following we will explain the scalar case first. Recall, that the forward pass of general permutohedral convolution involves 3 steps: splatting, convolving and slicing. We follow the same splatting and slicing strategies as in [1] since these operations do not depend on the filter kernel. The main difference between our work and the existing implementation of [1] is the way that the convolution operation is executed. This proceeds by constructing a blur neighbor matrix K that stores for every lattice point all values of the lattice neighbors that are needed to compute the filter output. The blur neighbor matrix is constructed by traversing through all the populated lattice points and their neighboring elements. This is done recursively to share computations. For any lattice point, the neighbors that are n hops (1, 1)